Information processing apparatus, information processing system, computer-readable non-transitory storage medium having stored therein information processing program, and information processing method

ABSTRACT

An example information processing apparatus including an infrared light emitter capable of emitting infrared light and a camera capable of at least taking an infrared light image is provided. In the information processing apparatus, first, an image taken by use of the camera is obtained. A face detection process based on the image is repeatedly executed. Then, based on a result of the face detection process, emission of the infrared light is stopped and emission of the infrared light is resumed after a lapse of a predetermined time period.

CROSS REFERENCE TO RELATED APPLICATION

The disclosure of Japanese Patent Application No. 2014-168527, filed onAug. 21, 2014, is incorporated herein by reference.

FIELD

The exemplary embodiments herein relate to an information processingapparatus including: an infrared light emitter capable of emittinginfrared light; and a camera capable of at least taking an infraredlight image, and more particularly, relate to an information processingapparatus capable of performing face detection based on at least aninfrared light image.

BACKGROUND AND SUMMARY

Conventionally, there has been known a technology in which infraredlight is emitted to a face, reflected light of the emitted infraredlight is detected to be outputted as an infrared light image, and facerecognition is performed by use of the infrared light image.

In the above technology, power consumption due to emission of infraredlight is not particularly taken into consideration. Thus, in a casewhere the above technology is applied to, for example, a hand-heldinformation terminal that operates on a battery, there is room forimprovement in terms of battery consumption used for emission ofinfrared light.

Therefore, an object of the present exemplary embodiments is to providean information processing apparatus, an information processing system, acomputer-readable non-transitory storage medium having stored thereinthe information processing program, and an information processingmethod, which are capable of reducing power consumption due to emissionof infrared light and at the same time capable of maintainingconvenience of face detection using infrared light.

In order to attain the above object, the following configurationexamples are conceivable, for example.

One example of the configuration example is an information processingapparatus including an infrared light emitter capable of emittinginfrared light and a camera capable of at least taking an infrared lightimage, the information processing apparatus configured to: controlemission of the infrared light; repeatedly obtain an image taken by useof the camera; and repeatedly execute a face detection process based onthe obtained image, wherein in the infrared light emission control,emission of the infrared light is stopped based on a result of the facedetection process performed in the face detection, and emission of theinfrared light is resumed after a lapse of a predetermined time period.

According to the above configuration example, while convenience by theface detection process is maintained, power consumption used in emissionof the infrared light can be suppressed.

As another configuration example, in the infrared light emissioncontrol, stopping emission of the infrared light and resuming emissionof the infrared light after a lapse of the predetermined time period maybe repeated.

According to the above configuration example, while power consumptionused in emission of the infrared light is further reduced, convenienceby the face detection process can be maintained.

As another configuration example, in the infrared light emissioncontrol, emission of the infrared light may be stopped when facedetection with the infrared light emitted has been failure, and emissionof the infrared light may be resumed after a lapse of the predeterminedtime period. Further, in the infrared light emission control, emissionof the infrared light may be stopped when the number of consecutivefailures of the face detection has become greater than a predeterminednumber, and emission of the infrared light may be resumed after a lapseof the predetermined time period.

According to the above configuration example, efficient powerconsumption reduction can be realized.

As another configuration example, in the infrared light emissioncontrol, stopping emission of the infrared light when face detectionwith the infrared light emitted has been failure and resuming theemission after a lapse of the predetermined time period may berepeatedly executed, until the face detection succeeds, with thepredetermined time period being gradually extended. Further, in theinfrared light emission control, as a result of the predetermined timeperiod being extended, when the predetermined time period has reached apredetermined value, the predetermined time period may not be extendedany longer and stopping emission of the infrared light and resuming theemission after a lapse of the predetermined time period may berepeatedly executed.

According to the above configuration example, while power consumption isfurther reduced, convenience by quick face detection can be maintained.

As another configuration example, the information processing apparatusmay be further configured to detect a motion of the informationprocessing apparatus itself, and

in the infrared light emission control, while stopping emission of theinfrared light and resuming the emission after a lapse of thepredetermined time period are repeatedly executed, when an output valueof the detected motion is greater than or equal to a predeterminedthreshold, emission of the infrared light may be resumed regardlesswhether the predetermined time period has elapsed.

According to the above configuration example, for example, in a darkplace, when a state where the user is not using the informationprocessing apparatus has changed to a state where the user is using theinformation processing apparatus, face detection can be quicklyperformed.

As another configuration example, the information processing apparatusmay be a hand-held information processing apparatus, or may be aninformation processing apparatus operable on a battery.

According to the above configuration example, while power consumption isreduced, convenience of the information processing apparatus can beenhanced.

According to the present exemplary embodiments, while power consumptionused in emission of the infrared light is reduced, convenience of theface detection process can be maintained.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an external view showing a non-limiting example of a hand-heldgame apparatus according to the present embodiment;

FIG. 2 is a block diagram showing a non-limiting example of aconfiguration of the hand-held game apparatus;

FIG. 3 shows a non-limiting example of shifts among control states in aprocess according to the present embodiment;

FIG. 4 is one example of programs and data stored in a main memory 35 ofa hand-held game apparatus 10;

FIG. 5 shows a non-limiting example of a flow chart showing details of aprocess performed in an OFF state;

FIG. 6 shows a non-limiting example of a flow chart showing details ofan ON Attempt determination process of step S2 shown in FIG. 5;

FIG. 7 shows a non-limiting example of a flow chart showing details of aprocess performed in an ON Attempt state;

FIG. 8 shows a non-limiting example of a flow chart showing details of aprocess performed in an ON state;

FIG. 9 shows a non-limiting example of a flow chart showing details ofan OFF Attempt determination process of step S32 in FIG. 8; and

FIG. 10 shows a non-limiting example of a flow chart showing details ofa process performed in an OFF Attempt state.

DETAILED DESCRIPTION OF NON-LIMITING EXAMPLE EMBODIMENTS

Hereinafter, one exemplary embodiment will be described. In the presentembodiment, description will be given while using a hand-held gameapparatus as one example of an information processing apparatus.

(Configuration of Hand-Held Game Apparatus)

FIG. 1 shows an external view of a hand-held game apparatus 10 accordingto the present embodiment. As shown in FIG. 1, the hand-held gameapparatus 10 includes a lower housing 11 and an upper housing 21. Thelower housing 11 and the upper housing 21 are connected to each other soas to be openable and closable in a folding manner (foldable). Usually,a user uses the hand-held game apparatus 10 in an opened state. When notusing the hand-held game apparatus 10, the user keeps the hand-held gameapparatus 10 in a closed state. The lower housing 11 is provided with alower LCD (liquid crystal display: liquid crystal display device) 12,and a touch panel 13. In addition, the lower housing 11 is also providedwith operation buttons, an analog stick, and the like. On the otherhand, the upper housing 21 is provided with an upper LCD (liquid crystaldisplay: liquid crystal display device) 22.

The hand-held game apparatus 10 includes a camera 101. The camera 101 isprovided (above the upper LCD 22 in FIG. 1) on a surface of the upperhousing 21. Thus, the camera 101 can take an image of the face of theuser present in front of the hand-held game apparatus 10. As oneexample, the camera 101 can take an image of the user in a state ofperforming a game while viewing the upper LCD 22 or the lower LCD 12.Further, the hand-held game apparatus 10 includes an infrared lightemitter 102 which emits infrared light. The infrared light emitter 102is provided at a position adjacent to the camera 101 on the surface ofthe upper housing 21. In other words, the infrared light emitter 102 isprovided at a position where it can emit infrared light toward aposition where the face of the user would be present while the user isusing the hand-held game apparatus 10 (such as while playing a game).

Here, the camera 101 can also take an infrared light image, in additionto a visible light image. Infrared light emitted from the infrared lightemitter 102 irradiates the face of the user, and reflected light thereofenters the camera 101. Based on the reflected light, the camera 101 cantake an infrared light image. In the present embodiment, based on animage taken by the camera 101 (visible light image or infrared lightimage), a process of detecting the face of the user (face detectionprocess) is executed. A result of the face detection is used as an inputfor a predetermined game. Moreover, a virtual camera is set based on thepositions of the eyes obtained as a result of the face detection, and aprocess of generating a virtual space image in accordance with theresult of the face detection, and the like are also executed.

(Internal Configuration of Hand-Held Game Apparatus)

FIG. 2 is a functional block diagram of the hand-held game apparatus 10.In FIG. 2, the hand-held game apparatus 10 includes an input device 31,a display device 32, a processor 33, an internal storage device 34, amain memory 35, an imaging section 36, a motion sensor 37, and theinfrared light emitter 102.

The input device 31 corresponds to the operation buttons and the touchpanel 13 mentioned above. The input device 31 is operated by the user ofthe hand-held game apparatus 10, and outputs a signal corresponding tothe operation made by the user. The display device 32 displays, on ascreen, an image generated in the hand-held game apparatus 10. Thedisplay device 32 is the lower LCD 12 and the upper LCD 22 describedabove. The internal storage device 34 has stored therein computerprograms to be executed by the processor 33 and various data to be usedin the programs. The internal storage device 34 is typically a flashEEPROM. In the main memory 35, computer programs and information aretemporarily stored. The motion sensor 37 is a sensor for detecting amotion made onto the housing itself of the hand-held game apparatus 10,and is an angular velocity sensor or an acceleration sensor, forexample. The imaging section 36 takes an image by use of the camera 101.The infrared light emitter 102 emits (turns on/turns off) infrared lightbased on an instruction from the processor 33.

It should be noted that, with respect to the processor 33, processesdescribed later may be performed by the single processor 33, oralternatively, the single hand-held game apparatus 10 is provided with aplurality of the processors 33, and the processes may be performed byuse of the plurality of the processors 33 in combination.

(Outline of Information Processing According to the Present Embodiment)

Next, the outline of operations of information processing (informationprocessing executed by the hand-held game apparatus 10) according to thepresent embodiment will be described. In the present embodiment, theface detection process and an infrared light emission control processare executed in parallel. Of these, for the face detection process, aknown technology is used, and detailed description thereof will beomitted in the description of the present embodiment. For example, byperforming pattern matching and the like based on a taken image, theface detection process is executed (then, based on a result of this facedetection, predetermined information processing is performed). As forthe “face detection” in the present embodiment, if the positions of theeyes have been detected, it is assumed that the “face detection” hasbeen successful. That is, in the present embodiment, it is sufficientthat the positions of the eyes are successfully detected.

Although the infrared light emission control process is also executed inparallel with the face detection process, the contents to be describedin the present embodiment are about the infrared light emission control.Hereinafter, the outline of the infrared light emission control processexecuted in the present embodiment will be described.

First, in the present embodiment, it is assumed to use infrared light,mainly, for taking an image of the face of the user in a state where thesurrounding of the hand-held game apparatus 10 is dark (a state whereambient light is insufficient). Therefore, typically, in a state wherethe surrounding of the hand-held game apparatus 10 is light (a statewhere ambient light is sufficient), the infrared light emitter 102 is ina state of “OFF” (not emitting infrared light), and in a dark state, theinfrared light emitter 102 is in a state of “ON” (emitting infraredlight). In the following description, the state of the infrared lightemitter 102 being ON will be referred to as a “turned-on state”, and thestate of the infrared light emitter 102 being OFF will be referred to asa “turned-off state”. Moreover, setting the infrared light emitter 102to be ON will be referred to as “turn on”, and setting the infraredlight emitter 102 to be OFF will be referred to as “turn off”.

Now, the following state is assumed. First, the user is using thehand-held game apparatus 10 in a dark room where no light is on.Further, face detection of the user has been successful. That is, theinfrared light emitter is on, and based on the infrared light, an imageof the face of the user has been taken by the camera 101, and facedetection has also been successful. In such a state, for example, a caseis considered where a light of the room is turned on. In this case,ambient light becomes sufficient and thus, face detection can beperformed without using infrared light. In such a case, if the infraredlight emitter 102 is kept on, unnecessary power will be consumedaccordingly. In particular, in a case where the hand-held game apparatus10 is operated on a battery, influence of power consumption due tounnecessary lighting of the infrared light emitter 102 is not small.Therefore, in the present embodiment, the following control is performedto reduce power consumption of the hand-held game apparatus 10. First,in a case where the infrared light emitter 102 is in the turned-onstate, and face detection has been successful (hereinafter, referred toas face detection success state), the infrared light emitter 102 is setto be the turned-off state once. Then, in this state, face detection isattempted. As a result, if face detection has failed, the infrared lightemitter 102 is returned to the turned-on state. On the other hand, ifface detection has been successful, control is performed such that theface detection process is continued while the infrared light emitter 102is kept in the turned-off state. In other words, while infrared light isin the turned-on state and face detection is being performed, infraredlight is turned off once and face detection is attempted. As a result ofthat, if face detection has failed, it is determined that the room isstill dark and it is a state where infrared light is necessary. On theother hand, if face detection has been successful, it is determined thatthe room having been dark has now become light, the hand-held gameapparatus 10 has moved from a dark place to a light place, or the like,and thus, it has become a state where infrared light need not be turnedon. By performing such control, in the present embodiment, it ispossible to prevent occurrence of a wasteful turned-on state of theinfrared light emitter 102, and to reduce power consumption.

In the present embodiment, the above control of turning off and turningon infrared light in the face detection success state is repeatedlyperformed. At this time, control of varying a time interval forattempting turning off infrared light is also performed. Specifically,the following control is performed: the interval (the time interval forturning off once) for switching the turned-on state to the turned-offstate is started from five seconds, and increased to be finally 60seconds. The reason for this is as follows. First, in a case where theuser is using the hand-held game apparatus 10 in a dark place(especially, when control of using the face detection process is beingperformed), if turning off of infrared light is frequently attempted,accuracy of face detection may be decreased, which may affect userexperience. On the other hand, unless turning off of infrared light isattempted periodically, even in a case where the hand-held gameapparatus 10 has moved from a dark place to a light place as describedabove, infrared light is kept on, and influence on power consumption isconcerned. Therefore, turning off of infrared light is attempted at afive second interval for some time at first, and then the attemptinterval of the turning off is gradually extended, whereby it isintended to prevent accuracy of face detection from decreasing and atthe same time, to reduce power consumption.

Further, in the present embodiment, the following control is alsoperformed. For example, the following state is assumed. First, the useris using the hand-held game apparatus 10 in a dark room and facedetection of the user has been successful. Then, a case is assumed inwhich, from this state, the user is no longer present in front of thehand-held game apparatus 10 by leaving the place or the like (the roomis still dark). In such a case, it becomes a state where detection ofthe face of the user will continue failing (hereinafter, referred to asface detection failure state). If the infrared light emitter 102 is kepton in this state, power will be continued to be consumed accordingly.Thus, in the present embodiment, control is performed such that in acase where the face detection failure state is continued to some extentin a dark room, the infrared light emitter 102 is turned off. Further,in the present embodiment, control is also performed such that evenafter the infrared light emitter 102 has been turned off, the infraredlight emitter 102 is turned on at a predetermined time interval.Accordingly, in a case where the user has returned to the front of thehand-held game apparatus 10, it becomes possible to quickly resume theprocess of detecting the face of the user. That is, in the presentembodiment, the following control is performed: when a state whereinfrared light has been turned on and face detection has been successfulhas changed to a state where failure of face detection continues,infrared light is turned off once, but thereafter, after a lapse of apredetermined time period, infrared light is turned on once and facedetection is attempted. As a result, if the detection has failed,infrared light is turned off again, and such control is repeated. On theother hand, if the detection has been successful, infrared light is kepton (because it is presumed that the user has returned after having leftthe place). Through such control, while reducing power consumption ofthe hand-held game apparatus 10, it is possible to prevent convenienceregarding face detection of the user from being impaired.

Further, with regard to the control in the case where the user has leftthe front of the hand-held game apparatus 10 in a dark room as above, inthe present embodiment, the following control is also performed. Asdescribed above, in a case where the face detection failure state hascontinued to some extent, infrared light is turned off once and after alapse of a predetermined time period, infrared light is turned on againto attempt face detection, and this procedure is repeated. In thisrepetition, control is performed such that the time interval betweensetting the turned-off state once and switching to the turned-on state(to attempt face detection with infrared light) is varied. Specifically,this time interval is started from 1/30 seconds, and increased to befinally three seconds. That is, in a case where the user is not presentin front of the hand-held game apparatus 10 in a dark place, theinfrared light emitter 102 is turned on (only for an instant) everythree seconds, eventually. This is for preventing the infrared lightemitter 102 from being wastefully turned on because too frequent attemptof turning it on results in increased power consumption. On the otherhand, there also is desire that the face detection success state isrealized as quickly as possible when the user has returned after havingleft the place. In order to realize both in good balance, in the presentembodiment, the control in which infrared light is turned on at aninterval of three seconds at maximum is performed as described above.

(Details of Information Processing According to the Present Embodiment)

Details of the infrared light emission control process according to thepresent embodiment will be described below. First, “(internal) controlstate” in the process will be described. In the present embodiment, thecontrol state includes the following four states.

(1) OFF state

(2) ON Attempt state

(3) ON state

(4) OFF Attempt state

FIG. 3 shows the four control states and shifts among the states. At thetime of activation of the hand-held game apparatus 10, the control stateis the “OFF state”.

First, (1) OFF state above is a state where the infrared light emitter102 is off. Main processing to be executed in this state is processingregarding determination of whether face detection has been successfuland determination of whether to turn on infrared light. This state canbe shifted to the ON Attempt state.

Next, (2) ON Attempt state is a state where the infrared light emitter102 is on. In this state, processing of determining whether facedetection has been successful and determining whether to turn offinfrared light or keep it on is mainly executed. This state can beshifted to the OFF state or the ON state.

Next, (3) ON state is a state where the infrared light emitter 102 ison. In this state, processing of determining whether face detection hasbeen successful and determining whether to turn off infrared light ismainly executed. This state can be shifted to the OFF state or the OFFAttempt state.

Next, (4) OFF Attempt state is a state where the infrared light emitter102 is off. In this state, processing of determining whether facedetection has been successful and determining whether to keep infraredlight off or turn on infrared light is mainly executed. This state canbe shifted to the OFF state or the ON state.

In the present embodiment, while the control state is shifted among thefour control states, the infrared light emission control process isrealized.

Next, main data to be used in the infrared light emission controlprocess according to the present embodiment will be described. FIG. 4shows one example of programs and data stored in the main memory 35 ofthe hand-held game apparatus 10. The main memory 35 has stored therein aface detection processing program 51, an infrared light emission controlprocessing program 52, a face detection state flag 53, an ON Attemptflag 54, an OFF Attempt flag 55, a detection-success counter 56, adetection-failure counter 57, a first threshold data 58, a secondthreshold data 59, a third threshold data 60, a fourth threshold data61, and the like.

The face detection processing program 51 is a program for executing theprocess of detecting the face of the user (the face detection process)based on an image taken by the camera 101. The infrared light emissioncontrol processing program 52 is a program for performing infrared lightemission control as described above. Specifically, the infrared lightemission control processing program 52 is a program for executing theprocesses of the flow charts shown in FIG. 5 to FIG. 10 described later.In the present embodiment, these programs are repeatedly executed every1/30 seconds (30 frames per second).

The face detection state flag 53 is a flag for indicating whethercurrent face detection has been successful or failure as a result of theface detection process. For example, “1” is set in the case of success,and “0” is set in the case of failure. As described above, in thepresent embodiment, the face detection process and the infrared lightemission control process are executed in parallel with each other. Then,data indicating the result of the face detection process is outputted asthe face detection state flag 53. In the infrared light emission controlprocess being executed in parallel, this flag is referred to, wherebywhether the current face detection has been successful can bedetermined.

The ON Attempt flag 54 is a flag to be used when determining, in the OFFstate, whether to make a shift to the ON Attempt state. The OFF Attemptflag 55 is a flag to be used when determining, in the ON state, whetherto make a shift to the OFF Attempt state. For each of the ON Attemptflag 54 and the OFF Attempt flag 55, when ON is set, it means thatmaking a shift is necessary, and when OFF is set, it means that making ashift is not necessary.

The detection-success counter 56 is a counter indicating the number ofconsecutive successes of face detection when infrared light is in theturned-on state. The detection-failure counter 57 is a counterindicating the number of consecutive failures of face detection. Asdescribed above, since the face detection process and the infrared lightemission control process are repeatedly executed every 1/30 seconds,increment by 1 is made to either one of the counters every 1/30 seconds.These counters are reset when a shift of the control state describedabove has occurred.

The first threshold data 58 is a threshold for determining, wheninfrared light is in the turned-off state, that infrared light is to beturned on. Specifically, the value of the detection-failure counter 57is compared with this threshold, and when the number of consecutivefailures of face detection has become greater than the first thresholddata, it is determined that infrared light is to be turned on. That is,the first threshold data 58 is a threshold for determining that infraredlight is to be turned on because the hand-held game apparatus 10 is in adark place. This threshold is mainly used for control in which “evenafter the infrared light emitter 102 has been turned off, the infraredlight emitter 102 is turned on at a predetermined time interval” asdescribed above. It should be noted that this threshold may be varied.Thus, in the present embodiment, as an initial value, a value indicating1/30 seconds is set. Moreover, in the present embodiment, a maximumvalue of the change in the first threshold data 58 is provided, and forexample, it is a value indicating a time period of three seconds.

The second threshold data 59 is a threshold for determining, in the ONAttempt state, that infrared light is to be turned off. Specifically,when the value of the detection-failure counter 57 has become greaterthan the threshold regardless of infrared light being on, it isdetermined that infrared light is to be turned off. For this threshold,a fixed value is set, and in the present embodiment, a valuecorresponding to five frames is set. The reason for this is as follows.After infrared light has been turned on, infrared light is notimmediately turned off because detection has been failure, but aboutfive frames are waited, whereby accuracy of face detection is furtherincreased.

The third threshold data 60 is a threshold for determining, wheninfrared light is in the turned-on state, that infrared light is to beturned off. Specifically, the value of the detection-success counter 56is compared with this threshold, and when the number of consecutivesuccesses of face detection has become greater than or equal to thethird threshold data, it is determined that infrared light is to beturned off. This threshold is mainly used for control in which “in thecase of the face detection success state, the infrared light emitter 102is set to be the turned-off state once” described above. This thresholdmay also be varied. In the present embodiment, as an initial value, avalue indicating a time period of five seconds is set. It should benoted that the third threshold data 60 is also provided with a maximumvalue of the change thereof. The maximum value is, for example, a valueindicating a time period of 60 seconds.

The fourth threshold data 61 is a threshold for determining, in a statewhere infrared light is in the turned-on state and face detection hasbeen failure, that infrared light is to be turned off. The value of thedetection-failure counter 57 is compared with this threshold, and whenthe number of consecutive failures is greater than the threshold, it isdetermined that infrared light is to be turned off. This threshold isused for control in which infrared light is turned off in such a casewhere the user has left the front of the hand-held game apparatus 10 ina dark place as described above. For this threshold, a fixed value isset, and, for example, a value corresponding to a time period of threeseconds is set. That is, in a case where the user has left the front ofthe hand-held game apparatus 10 in a dark place, infrared light is to beturned off once, after about three seconds.

Other than the above, although not shown, various data to be used inprocesses according to the present embodiment is also stored in the mainmemory 35 as appropriate.

Next, a detailed flow of the infrared light emission control processaccording to the present embodiment will be described. FIG. 5 to FIG. 10are flow charts showing details of the processes. FIG. 5 and FIG. 6 showcontrol processes to be performed in the OFF state described above. FIG.7 shows a control process to be performed in the ON Attempt state. FIG.8 and FIG. 9 show processes to be performed in the ON state. FIG. 10shows a process to be performed in the OFF Attempt state. It should benoted that the processes of the flow charts in the respective states arerepeatedly executed every 1/30 seconds. Moreover, as described above, inparallel with execution of these processes, the face detection processis repeatedly (every 1/30 seconds) executed. As the face detectionprocess itself, a known technology is employed, and detailed descriptionthereof is omitted.

(Information Processing in OFF State)

First, a process performed when the control state is the OFF state willbe described with reference to FIG. 5 and FIG. 6. In this state, first,in step S1 of FIG. 5, the processor 33 refers to the face detectionstate flag 53 and determines whether face detection has been successfulat that time point. As a result, when face detection has been successful(YES in step S1), it is not necessary to turn on infrared light, andthus, determination in step S1 will be repeated (i.e., the current stateis maintained).

On the other hand, face detection has been failure (NO in step S1), theprocessor 33 adds 1 to the detection-failure counter 57. Then, in stepS2, the processor 33 executes an ON Attempt determination process fordetermining whether to turn on infrared light. FIG. 6 is a flow chartshowing details of the ON Attempt determination process. First, in stepS11, the processor 33 determines whether the hand-held game apparatus 10is in sufficient ambient light. This can be determined, for example, bydetermining brightness of light incident on the camera 101, or the like.In another embodiment, a brightness sensor dedicated to detection ofbrightness may be provided in the hand-held game apparatus 10.

As a result of the determination, when it has been determined that thehand-held game apparatus 10 is in sufficient ambient light (YES in stepS11), it is considered that turning on of infrared light is notnecessary. In this case, in step S17, the processor 33 sets the ONAttempt flag 54 to Off. Then, the ON Attempt determination process ends.On the other hand, when it has been determined that the hand-held gameapparatus 10 is not in sufficient ambient light (NO in step S11), then,in step S12, the processor 33 refers to the detection-failure counter 57and the first threshold data 58, and determines whether the number ofconsecutive failures of face detection is greater than the firstthreshold. As a result, when the number of consecutive failures of facedetection is not greater than the first threshold (NO in step S12), theprocess is advanced to step S17. That is, until the time periodindicated by the first threshold has elapsed, the face detection processwill be continued with infrared light in the turned-off state.

On the other hand, the number of consecutive failures of face detectionis greater than the first threshold (YES in step S12), next, in stepS13, the processor 33 determines whether there is indication that theuser is present in front of the hand-held game apparatus 10. That is, itis determined whether, currently, the user is using the hand-held gameapparatus 10 or has left the place, leaving the hand-held game apparatus10 there. In the present embodiment, this determination is made based ondata outputted from the motion sensor 37, whether a predetermined buttonoperation is being performed, or the like. For example, in a case whereit can be determined, based on data from the motion sensor 37, thatmotion of the hand-held game apparatus 10 itself is occurring, it isconsidered that the user is holding the hand-held game apparatus 10 withhands and is operating it. Moreover, also in a case where apredetermined button input can be detected, it can be presumed that theuser is using the hand-held game apparatus 10. As a result of suchdetermination, when there is indication that the user is present infront of the hand-held game apparatus 10 (it is considered that the useris currently using the hand-held game apparatus 10) (YES in step S13),it is considered that the user is using the hand-held game apparatus ina dark place, for example. Thus, it is necessary to immediately turn oninfrared light. In this case, in step S14, the processor 33 resets thefirst threshold data 58 to the initial value. Next, in step S15, theprocessor 33 executes a process for increasing the value of the firstthreshold data 58. Specifically, the processor 33 calculates a new firstthreshold data 58 by use of the following formula.

First threshold=(First threshold+1)×1.5  formula 1

It should be noted that with respect to the process of step S15, themaximum value (in this example, a value indicating a time period ofthree seconds) is set for the first threshold data 58 as describedabove. Thus, as a result of the above calculation, when the maximumvalue is reached, calculation thereafter will not be performed.

On the other hand, in the determination in step S13, when it has beendetermined that there is no indication that the user is present in frontof the hand-held game apparatus 10 (NO in step S13), the process of stepS14 described above is skipped and the process is advanced to step S15.That is, the first threshold data 58 is not reset, and the value thereofwill be increased.

Next, in step S16, the processor 33 sets the ON Attempt flag 54 to ON.Then, the ON Attempt determination process ends.

With reference back to FIG. 5, next, in step S3, the processor 33 refersto the ON Attempt flag 54 and determines whether to make a shift to theON Attempt state. As a result, when the ON Attempt flag 54 is OFF (NO instep S3), the OFF state will be maintained, and the process is returnedto step S1 to be repeated.

On the other hand, when the ON Attempt flag 54 is ON (YES in step S3),then, in step S4, the processor 33 turns on the infrared light emitter102. Then, in step S5, the processor 33 executes a process for making ashift to the ON Attempt state. At this time, the detection-successcounter 56 and the detection-failure counter 57 are also reset. This isthe end of description of the processes performed in the OFF state.

(Information Processing in ON Attempt State)

Next, a process to be performed in the ON Attempt state will bedescribed with reference to FIG. 7. First, in step S21, the processor 33refers to the face detection state flag 53 and determines whether facedetection has been successful at that time point. As a result, when facedetection has been successful (YES in step S21), then, in step S22, theprocessor 33 executes a process for making a shift to the ON state. Atthis time, the detection-success counter 56 and the detection-failurecounter 57 are reset, and the first threshold data 58 is also reset.

On the other hand, when face detection has been failure (NO in stepS21), then, in step S23, the processor 33 refers to thedetection-failure counter 57 and the second threshold data 59, anddetermines whether the value of the detection-failure counter 57 isgreater than the second threshold data 59. As a result, when the valueof the detection-failure counter 57 is not greater than the secondthreshold data 59 (NO in step S23), the process is returned to step S21to be repeated. In the present embodiment, as the second threshold data59, a value indicating a time period corresponding to five frames is setas a fixed value. Thus, here, the face detection process will berepeated for the time period corresponding to five frames.

On the other hand, when the value of the detection-failure counter 57 isgreater than the second threshold data 59 (YES in step S23), it meansthat face detection is consecutively failing regardless of infraredlight being on (for the time period corresponding to five frames orlonger). Thus, in step S24, the processor 33 turns off the infraredlight emitter 102, and in step S25, executes a process of making a shiftto the OFF state. This is the end of description of the processperformed in the ON Attempt state.

(Information Processing in ON State)

Next, a process performed in the ON state will be described withreference to FIG. 8 and FIG. 9. First, in step S31 of FIG. 8, theprocessor 33 refers to the face detection state flag 53 and determineswhether face detection has been successful at that time point. As aresult, when face detection has been successful (YES in step S31), then,in step S32, the processor 33 executes an OFF Attempt determinationprocess. At this time, the processor 33 adds 1 to the detection-successcounter 56.

FIG. 9 is a flow chart showing details of the OFF Attempt determinationprocess. In this process, first, it is determined whether the relativepositional relationship between the user and the hand-held gameapparatus 10 is stable. In the present embodiment, the followingdetermination is performed. First, in step S41, the processor 33determines whether a motion having a magnitude greater than or equal toa predetermined threshold is occurring in the hand-held game apparatus10 itself. This is determined based on, for example, whether an outputfrom the motion sensor 37 being an angular velocity sensor, anacceleration sensor, or the like is greater than or equal to apredetermined threshold. That is, it is determined whether the user isholding the hand-held game apparatus 10 and moving it vigorously, or thelike. As a result of this determination, when a motion greater than orequal to the predetermined threshold is occurring (YES in step S41), itis presumed that the relative positional relationship between the userand the hand-held game apparatus 10 is greatly changing. Here, when therelative positional relationship between the user and the hand-held gameapparatus 10 is greatly changing, there is a possibility that accuracyof face detection is decreased. When taking this into consideration, inorder to perform accurate face detection, it is preferable that infraredlight is kept on. Therefore, in this case, in step S46, the processor 33sets the OFF Attempt flag 55 to OFF.

On the other hand, as a result of the above determination, when nomotion greater than or equal to the predetermined threshold is occurring(NO in step S41), next, in step S42, the processor 33 determines whetherthere is motion in the face of the user. This is assuming such a casewhere the user is playing while the hand-held game apparatus 10 itselfis placed on a desk, for example. Even when no motion is occurring inthe hand-held game apparatus 10 itself (or even when a very small motionis occurring), if a motion is recognized in the face itself of the user(in this example, the positions of the eyes of the user), it is alsopresumed that the relative positional relationship between the user andthe hand-held game apparatus 10 is greatly changing. This determinationcan be made based on, for example, whether the position (the centerposition between the eyes) of the face of the user obtained as a resultof the face detection process has moved some distance or more from aposition detected in the immediately preceding frame. Alternatively, thedetermination can be also made, for example, by a technique ofdetermining whether a change in the position of the face of the user ina period of most recent several frames to several tens of frames isgreat to some extent.

In other words, the determination in each of step S41 and S42 isdetermination of the magnitude of a change in relative positionalrelationship between the user and the hand-held game apparatus 10 (thecamera 101). Therefore, not being limited to the above determinationtechnique, as long as the magnitude of a change in the relativepositional relationship can be determined, any technique may be used.

As a result of the determination in step S42, when a motion is occurringin the face of the user (YES in step S42), the process is advanced tostep S46 described above. That is, at this time point, it is determinedthat infrared light is not to be turned off. The reason for this is asfollows: in a state where the user is presumed to be present in front ofthe hand-held game apparatus 10, if infrared light is turned off whenthere is a large change in the relative positional relationship betweenthe user and the hand-held game apparatus 10, accuracy of face detectionis decreased. On the other hand, when no motion is occurring (NO in stepS42), next, in step S43, the processor 33 refers to thedetection-success counter 56 and the third threshold data 60, anddetermines whether the number of consecutive successes of face detectionis smaller than the third threshold. When the number of consecutivesuccesses is smaller than the third threshold (YES in step S43), controlis performed such that turning off of infrared light is not attempted.Thus, the process is advanced to step S46 described above. On the otherhand, when the number of consecutive successes is not smaller than thethird threshold (i.e., greater than or equal to the third threshold) (NOin step S43), control is performed for attempting determination ofwhether face detection can be successful even when infrared light isturned off because the state of the surrounding has changed from a darkstate to a light state, or the like. In this case, in step S44, theprocessor 33 executes a process for increasing the value of the thirdthreshold data 60. Specifically, the processor 33 calculates a new thirdthreshold data 60 by use of the following formula.

Third threshold=(Third threshold+1)×1.5  formula 2

It should be noted that with respect to the process of step S44, themaximum value (in this example, a value indicating a time period of 60seconds) is set to the third threshold data 60 as described above. Thus,as a result of the above calculation, when the maximum value is reached,calculation thereafter will not be performed.

Next, in step S45, the processor 33 sets the OFF Attempt flag 55 to ON.Then, the OFF Attempt determination process ends.

With reference back to FIG. 8, next, in step S33, the processor 33refers to the OFF Attempt flag 55 and determines whether to make a shiftto the OFF Attempt state. As a result of the determination, when the OFFAttempt flag 55 is OFF (NO in step S33), the ON state is maintained.Thus, the process is returned to step S31 to be repeated.

On the other hand, when the OFF Attempt flag 55 is ON (YES in step S33),then, in step S34, the processor 33 turns off the infrared light emitter102. Then, in step S35, the processor 33 executes a process for making ashift to the OFF Attempt state. At this time, the detection-successcounter 56 and the detection-failure counter 57 are also reset. Then, inthe OFF Attempt state, whether face detection in the turned-off state issuccessful will be determined (the process to be performed in the OFFAttempt state will be described later).

Next, description will be given of a process to be performed in a casewhere, as a result of the determination in step S31, face detection hasbeen failure (NO in step S31). In this case, first, in step S36, theprocessor 33 refers to the detection-failure counter 57 and the fourththreshold data 61, and determines whether the number of consecutivefailures of face detection is smaller than the fourth threshold. Thatis, it is determined, for example, whether it is a state where the userhas left the front of the hand-held game apparatus 10 in a dark room. Inthis example, since the fourth threshold is set as a value indicating atime period of three seconds, if failure of face detection continues forthree seconds or longer, the condition for the determination issatisfied.

As a result of the determination, when the number of consecutivefailures is not greater than the fourth threshold (NO in step S36), theprocess is returned to step S31 to be repeated. On the other hand, whenthe number of consecutive failures is greater than the fourth threshold(YES in step S36), then, in step S37, the processor 33 turns off theinfrared light emitter 102. Then, in step S38, the processor 33 executesa process for making a shift to the OFF state (as in the above, at thetime of making a shift, relevant counters are reset). This is the end ofdescription of the process performed in the ON state.

(Information Processing in OFF Attempt State)

Next, a process to be performed in the OFF Attempt state will bedescribed with reference to FIG. 10. First, in step S51 of FIG. 10, theprocessor 33 refers to the face detection state flag 53 and determineswhether face detection has been successful at that time point. As aresult, when face detection has been successful (YES in step S51), it isnot necessary to turn on infrared light, and thus, in step S52, theprocessor 33 executes a process for making a shift to the OFF state. Onthe other hand, when face detection has been failure (NO in step S51),then, in step S53, the processor 33 turns on the infrared light emitter102 in order to attempt face detection in a state where infrared lightis on. Then, in step S54, the processor 33 executes a process for makinga shift to the ON state. Then, the process performed in the OFF Attemptstate ends.

With respect to the processes of steps S51 and S52, the followingcontrol may be performed. That is, in turning off the infrared lightemitter, assuming that the turning off is performed gradually after aninstruction of the turning off has been made, when the process is to beshifted from step S51 to step S52, a time period of several frames iswaited (it is waited until the infrared light emitter is completelyturned off), and then, the process may be advanced to step S52.Specifically, when the determination in step S51 is YES, a process isperformed of determining whether a predetermined number of frames (atime period corresponding to the predetermined number of frames) haveelapsed after the instruction of the turning off has been made. As aresult of this determination, when the time period has not elapsed, theprocess is returned to step S51, and when the time period has elapsed,the process is advanced to step S52.

Through the processes described above, for example, the followingcontrol is performed. First, in a state where face detection has beensuccessful in the ON state, a shift to the OFF Attempt state is(forcedly) made, once. The interval of this shift starts from fiveseconds, and then gradually extended, to be finally 60 seconds. Next, itis determined whether face detection can be successful in the OFFAttempt state. As a result, when face detection is not successful, it ispresumed that it is a state where infrared light is necessary (forexample, the surrounding is dark), and thus, a shift to the ON state ismade. On the other hand, when face detection has been successful, it ispresumed that it is a state where infrared light is unnecessary (forexample, the surrounding is light), and thus, a shift to the OFF stateis made.

Furthermore, the following control is also performed. For example, whenface detection consecutively fails to some extent in the ON state, it ispresumed that the user has left the place and a shift to the OFF stateis made once. Next, face detection is repeated for a predetermined timeperiod in the OFF state, and when the predetermined time period haselapsed, a shift to the ON Attempt state is made once. The predeterminedtime period starts from 1/30 seconds, to be finally three seconds. Next,it is determined whether face detection has been successful in the ONAttempt state. When face detection has been failure, it is presumed thatthe user is still away from the place, and thus, a shift to the OFFstate is made. On the other hand, when face detection has beensuccessful, it is presumed that the user has returned to the place, andthus, a shift to the ON state is made.

As described above, in the present embodiment, even when face detectionhas been successful, turning off of infrared light is attempted, toprevent infrared light from unnecessary being on, whereby powerconsumption is to be reduced. Moreover, in a case where the user hasleft the front of the hand-held game apparatus 10 when the surroundingis dark, infrared light is turned off once, and thereafter, infraredlight is periodically turned on, whereby countermeasures for a casewhere the user has returned are also taken. Through this control, it isintended to realize reduction of power consumption and maintainingconvenience for the user by the face detection process. In particular,if the above-described control is applied to an information processingapparatus driven on a battery, effect of reduction of power consumptioncan be exhibited to a great extent.

In the above embodiment, control of turning off infrared light isperformed when the user has left in a dark place. However, at this time,control in which hardware components other than infrared light are setto OFF may be performed. For example, control may be performed such thatthe upper LCD 22 and the lower LCD 12 are set to OFF associated withturning off of infrared light. Accordingly, further reduction of powerconsumption can be realized.

In the above embodiment, with respect to success/failure of facedetection, the determination process using “the number” ofsuccesses/failures is performed. Other than “the number”, a “timeperiod” may be used to perform the determination. For example, thedetermination may be performed based on the time period of consecutivesuccesses, or the time period of consecutive failures.

In the above example, the hand-held game apparatus 10 is used as oneexample of the information processing apparatus. Other than this, thecontrol described in the above embodiment can be applied to varioushand-held information processing apparatuses (such as smartphones),tablet terminals, notebook computers, and the like, in particular,various information processing apparatuses operable on batteries.

In the above embodiment, a case has been described in which a series ofprocesses for controlling emission of infrared light of the hand-heldgame apparatus 10 is executed by a single apparatus. However, in otherembodiments, the series of processes may be executed in an informationprocessing system including a plurality of information processingapparatuses. For example, in an information processing system includinga terminal side apparatus and a server side apparatus capable ofcommunicating with the terminal side apparatus via a network, one partof the series of processes may be executed on the server side apparatus.Further, in an information processing system including a terminal sideapparatus and a server side apparatus capable of communicating theterminal side apparatus via a network, main processes of the series ofprocesses may be executed on the server side apparatus, and one part ofthe processes may be executed on the terminal side apparatus. Stillfurther, in the information processing system described above, theserver side system may be configured with a plurality of informationprocessing apparatuses, and processes to be executed on the server sidemay be divided to be executed by the plurality of information processingapparatuses.

What is claimed is:
 1. An information processing apparatus including aninfrared light emitter capable of emitting infrared light and a cameracapable of at least taking an infrared light image, the informationprocessing apparatus configured to: control emission of the infraredlight; repeatedly obtain an image taken by use of the camera; andrepeatedly execute a face detection process based on the obtained image,wherein in the infrared light emission control, emission of the infraredlight is stopped based on a result of the face detection processperformed in the face detection, and emission of the infrared light isresumed after a lapse of a predetermined time period.
 2. The informationprocessing apparatus according to claim 1, wherein in the infrared lightemission control, stopping emission of the infrared light and resumingemission of the infrared light after a lapse of the predetermined timeperiod are repeated.
 3. The information processing apparatus accordingto claim 1, wherein in the infrared light emission control, emission ofthe infrared light is stopped when face detection with the infraredlight emitted has been failure, and emission of the infrared light isresumed after a lapse of the predetermined time period.
 4. Theinformation processing apparatus according to claim 3, wherein in theinfrared light emission control, emission of the infrared light isstopped when the number of consecutive failures of the face detectionhas become greater than a predetermined number, and emission of theinfrared light is resumed after a lapse of the predetermined timeperiod.
 5. The information processing apparatus according to claim 3,wherein in the infrared light emission control, stopping emission of theinfrared light when face detection with the infrared light emitted hasbeen failure and resuming the emission after a lapse of thepredetermined time period are repeatedly executed, until the facedetection succeeds, with the predetermined time period being graduallyextended.
 6. The information processing apparatus according to claim 5,wherein in the infrared light emission control, as a result of thepredetermined time period being extended, when the predetermined timeperiod has reached a predetermined value, the predetermined time periodis not extended any longer and stopping emission of the infrared lightand resuming the emission after a lapse of the predetermined time periodare repeatedly executed.
 7. The information processing apparatusaccording to claim 1, wherein the information processing apparatus isfurther configured to detect a motion of the information processingapparatus itself, and in the infrared light emission control, whilestopping emission of the infrared light and resuming the emission aftera lapse of the predetermined time period are repeatedly executed, whenan output value of the detected motion is greater than or equal to apredetermined threshold, emission of the infrared light is resumedregardless whether the predetermined time period has elapsed.
 8. Theinformation processing apparatus according to claim 1, wherein theinformation processing apparatus is a hand-held information processingapparatus.
 9. The information processing apparatus according to claim 1,wherein the information processing apparatus is an informationprocessing apparatus operable on a battery.
 10. An informationprocessing system including an infrared light emitter capable ofemitting infrared light and a camera capable of at least taking aninfrared light image, the information processing system configured to:control emission of the infrared light; repeatedly obtain an image takenby use of the camera; and repeatedly execute a face detection processbased on the obtained image, wherein in the infrared light emissioncontrol, emission of the infrared light is stopped based on a result ofthe face detection process performed in the face detection, and emissionof the infrared light is resumed after a lapse of a predetermined timeperiod.
 11. A computer-readable non-transitory storage medium havingstored therein an information processing program to be executed by acomputer of an information processing apparatus including an infraredlight emitter capable of emitting infrared light and a camera capable ofat least taking an infrared light image, the information processingprogram causing the computer to: control emission of the infrared light;repeatedly obtain an image taken by use of the camera; and repeatedlyexecute a face detection process based on the obtained image, wherein inthe infrared light emission control, emission of the infrared light isstopped based on a result of the face detection process performed in theface detection, and emission of the infrared light is resumed after alapse of a predetermined time period.
 12. An information processingmethod for controlling an information processing apparatus including aninfrared light emitter capable of emitting infrared light and a cameracapable of at least taking an infrared light image, the methodcomprising: controlling emission of the infrared light; repeatedlyobtaining an image taken by use of the camera; and repeatedly executinga face detection process based on the obtained image, wherein in theinfrared light emission control, emission of the infrared light isstopped based on a result of the face detection process performed in theface detection, and emission of the infrared light is resumed after alapse of a predetermined time period.